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Abstract: The aim of this paper is to explore the possibilities 
of Conley index techniques in the study of heteroclinic connec- 
tions between finite and infinite invariant sets. For this, we 
remind the reader of the Poincare compactification: this trans- 
formation allows to project a n-dimensional vector space X on 
the n-dimensional unit hemisphere of X x R and infinity on its 
(n — l)-dimensional equator called the sphere at infinity. Under 
a normalizability condition, vector fields on X are mapped to 
vector fields on the Poincare hemisphere whose associated flows 
leave the equator invariant. The dynamics on the equator re- 
flects the dynamics at infinity, but are now finite and may be 
studied by Conley index techniques. Furthermore, we observe 
that some non-isolated behavior may occur around the equa- 
tor, and introduce the concept of an invariant set at infinity 
of isolated invariant dynamical complement. Through the con- 
struction of an extended phase space together with an extended 
flow, we are able to adapt the Conley index techniques and prove 
the existence of connections to such non-isolated invariant sets. 

1 Introduction 

The analysis of the structure of connections in global attractors inspired several tools, in partic- 
ular the Conley index techniques. Phenomena like blow up or grow up may be interpreted as 
heteroclinic connections to infinity. Examples for such situations are to be found in [101112] 
In this paper, we describe a way of using the Conley index theory for the analysis of unbounded 
global attractors - or more generally unbounded invariant sets. For this, we need to analyse both 
the finite connections and those going to infinity in forward or in backward time. 
The first step of our method consists in making infinity finite by Poincare compactification. The 
phase space is projected to a hemisphere of the same dimension, infinity being projected onto its 
equator. The equator is the boundary of the hemisphere and is itself a sphere called the sphere 
at infinity. Its dimension is (dimension of the phase space) — 1. For more details see Section [5] 
and in particular Figure [TJ This procedure grants us two things: the exhibition of the relevant 
dynamic at infinity, i.e. the invariant sets in the sphere at infinity, and the consideration of 
bounded neighbourhoods of these. 

However, Conley index does not only require bounded neighborhoods of invariant sets, but iso- 
lating ones. For more details on classical Conley index theory, see the original work [J and 
good introductions in [5Jini[IZ|. In the question of isolation, it turns out that infinity, although 
compactified, still stands out by virtue of its lack of isolation in many cases. To illustrate this, let 
us have a look at Figure O it shows equilibria at infinity that are not isolated, in phase portraits 
18 and 39. 

To circumvent this difficulty, we introduce the notion of a complementary isolated invariant set. 
The precise definition will be given in 13.11 Let us here illustrate the three possible situations 
with three examples. We pick up those examples in [2j[T6] where classifications of (quadratic) 



1 



planar vector fields are given. The numbers associated to those phase portrait coincide with 
their numbers in the papers. 

1. The phase Portrait 25 of Figure @ is from [5]. It admits only isolated invariant equilibria 
at infinity. 

2. The phase Portrait 18 of Figure (jSl) is from [2]. It admits an equilibrium at the bottom 
which is not isolated invariant but of isolated invariant dynamical complement. 

3. The phase Portrait 39 of Figure ([3]) comes from [16]. It admits an equilibrium at the 
bottom which is neither isolated invariant nor complementarily isolated. 

A Conley index at infinity for such invariant sets is defined and denoted by h{.). This index 
is based on time-duality and the classical index of the dynamical complement - an interest- 
ing new concept that we introduce in Definition 13.11 .With its help, one can state basic exis- 
tence/nonexistence theorems of the following type (see Theorem p.l6[) for more details, as well 
as Definition 13. ISp . 

Theorem. Consider an invariant set S at infinity of isolated invariant complement. If the 
Conley index at infinity of S, h{S), is neither of a repeller nor of an attractor, then there are 
trajectories accumulating on S in forward time direction, and trajectories accumulating on S in 
backward time direction. 

Furthermore we introduce a construction in order to analyse the connection structure of an 
unbounded invariant set, provided it consists of isolated invariant parts or of invariant parts of 
isolated invariant complement. The rough idea is to replace the parts that are invariant and only 
complementarily isolated by an ersatz infinity. This will be done in Section [4.2l After proceeding 
with this construction, we are able to prove the main Theorem 14.81 The content of this theorem 
can be summarized as follows. 

Theorem. // the Conley index theory proves the existence of connecting orbits from/to the 
isolated invariant ersatz infinity to/from an isolated invariant set R C Scomp for the extension 
% with extended flow (j), 

then the corresponding connecting orbits S — !■ R/R — > S exist for the original compactified flow 
(j) on the Poincare Hemisphere Ti. 

Hence we are able to detect heteroclinic orbits to sets which, a priori, were beyond the reach 
of Conley index theory. Such invariant sets pop up naturally when studying transfinite dynamics 
and the connection structure of unbounded invariant sets. 

The author thanks the referee for his constructive remarks, and Ahmad Afuni for his careful 
corrections reading. This research has been supported by the Deutsche Forschungsgcmeinschaft, 
SFB 647-Space- Time-Matter. 

2 Poincare compact ificat ion 

The Poincare compactification transforms a finite dimensional vector space X into a compact 
manifold - a hemisphere - and is therefore called a compactification. The construction also 
works for infinite dimensional Hilbert spaces X, in which case the image would be an infinite 
dimensional hemisphere, which is a noncompact Hilbert manifold with boundary. Arbitrarily 
far points in the vector space are mapped onto the equator - the boundary of the Poincare 
hemisphere. Therefore we will also refer to it as the sphere at infinity. The sphere at infinity is 
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Figure 1: The Poincare transformation. 



a sphere whose dimension reads dim(X) — 1. We refer to the overview paper [TU], and to the 
books [TlfH] where exphcit planar examples are computed. Now, let us describe the construction 
of the Poincare compactification more precisely. 

Let X be an Hilbert space whose scalar product is denoted by (•,•). We consider the vector 
space X X ]R. In other words, we add to X a, say, vertical dimension whose coordinate will 
be denoted by z G M. The vector space X x IR is naturally equipped with the scalar product 
((x, z), (x', z')) := {x, x')+zz' . The original vector space X is identified with the affine hyperplane 
X X {1} through the bijcction x <E X t-^ {x,l) <E X x {1}. Finally, the afSnc hyperplane X x {1} 
is projected centrally to the unit hemisphere 

n := {(x, z) (E X X R/{x, x) + z^ = I, z ^ 0}, 

called the Poincare hemisphere. This transformation is sketched in Figure 
The Poincare transformation is given by the formula 

V : X ^ H 

(x,2) = ^===(a;, 1). 

+ 1 

As 1 1 a: 1 1 tends to infinity, V{x) goes to the equator 

£ ■.^{{x,o)ex xR/ (x,x) = l} 

of the Poincare hemisphere. Hence the equator £ is also called the sphere at infinity. 

Now let us describe how vector fields on the vector space X are mapped under the Poincare 
transformation. For this purpose, let us consider the equation 

x^fix), xex, (1) 

where the upper dot denotes time derivative. Applying the Poincare transformation, we get the 
following evolution equations for the image (x, z) = V{x), z > 0: 

i = ^{Mx),x)z 
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where denotes the homothctic of / with factor z; in other words 

Equation ^ makes sense on the sphere at infinity if and only if has a Hmit as z tends to zero. 
This is rarely the case. However this difficulty can be overcome by normalization: we replace fz 
by p{z)fz in Equation ([5]), where p is 

1. a positive function, 

2. strictly positive on H \ £ so that the finite trajectories are not affected, 

3. tends quickly enough to zero as z — > 0, slowing down the trajectories and guaranteeing the 
existence of a continuous limit: 

\imp{z)fz{x) /o(x)- 

2— >0 

More precisely, normalizable vector fields are defined as follows. 
Definition 2.1. Normalization of the homothety 

The vector field f : X ^ X la normalizable if and only if there exists a continuous function 

p : [0, +oo[— !• [0, +cxd[ such that 

1. p{z) 7^ if z 7^ 0, and 

2. the map (x, z) i— > p{z)fz{x) is Lipschitz continuous even at z = 0. 
Remark 2.2. Normalization is a change of time variable. 

Replacing the homothety fz{x) by its normalized version p{z)fz{x) corresponds to a change of 
time variable dt = p{z)dT. We will nevertheless keep the name t for the time variable of the 
normalized equations on the Poincare hemisphere for the sake of simplicity. When we speak 
of a flow/vector field on the Poincare hemisphere, we mean such a normalized equation where 
regularity all the way up to the equator is given. 

Remark 2.3. Vector fields with polynomial growth are normalizable. Consider a map 
f : X ^ X admitting a decomposition of the following type: 

.f = P + P. 

where P is homogenous of degree d ( i. c. P{\x) = X'^P{x)) and p contains terms of order lower 
than d — 1, but not necessarily polynomial (i. e. p{x) = o(||a:||'') for |la;|| going to infinity). The 
normalization by p{z) = z"^"^ provides 

Hm p{z)fz = P. 

We will from now on mostly omit mention of normalization, and speak only of compactified 
vector field/flow on the Poincare hemisphere H, meaning in fact compactified and normalized. 
Otherwise we do not even get a proper flow on H. 

At last, we give some more formulas which are useful for computations. Instead of analyzing 
a vector field near the equator of the Poincare hemisphere (typically, looking for its equilibria 
at infinity and their stability), it is easier to compute in vertical hyperplanes tangent to the 
hemisphere around the region of interest. Therefore, we need the following projection Projp onto 
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the vertical affinc half hyperplane E tangent to H at e G i. e. E = (e+ < e n(X x [0, oo[). 
We call the coordinates in these planes the "vertical charts" . 

Proje ■■ H E 
The resulting vector field on E in the variables C) reads 



i = -(/c(0,e)C + /c(0 
C = -(/c(0,e)C 



(3) 



Here again, it is often necessary to replace by its normalized form pf^ to realize regularity of 
the projected vector field on the vertical charts. 

After compactification, the sphere at infinity may contain isolated invariant sets whose Conley 
indices are well defined. We recall the definition of the Conley index for an isolated invariant set 
intersecting the boundary of the Poincare hemisphere: it splits into three components because 
the standard definition is not sufficient to take the dynamics on the boundary 8% = £ into 
consideration. This definition coincides with those of [TTJ[T3]. 

Let us first fix some vocabulary. The property of being open - or closed - is to be understood 
relative to the Poincare hemisphere %. We write, when possible, a subscript H to remind the 
reader of this. Furthermore, a set K is said to be a compact neighborhood if K is compact and 
the closure of its interior K = cl-u{int-u{K)) . An (forward and backward) invariant set S is said 
to be isolated invariant iff there is a compact neighborhood K oi S for which S is the maximal 
invariant set Inv{K) = S and S C int-uiK)- 

Definition 2.4. Conley index on the boundary. 

We consider an isolated invariant set S on the Poincare hemisphere % with boundary £. The 
set S admits an index pair {N,Ni), i.e. a pair of compact set satisfying 

1. Isolation: The set cl{N \ Ni) is an isolating neighbourhood for S. More precisely, S C 
d{N \Ni) does not intersect the boundary of clu{N \ Ni) relatively to T-L. 

2. Positive invariance of A^i with respect to N: A trajectory starting in A^i remains in 
A^i until it leaves N, i. e. for all x £ Ni and all t > 0, .^([0, t],x) C N =^ ^([0, t],x) C Ni. 

3. The set A^i is an exit set for N: The trajectories leave N through A^i, i. e. for all x G N 
and ti > with ifi{ti,x) ^ N, there exists a time to £ [0,ti], such that ip{[0,to], x) C N 
and ip{tQ,x) E Ni . 

Let * ^ X he a. universal point. The Conley index of S is defined by three homotopy classes of 
pointed spaces as follows: 



h{n;S) 



jvu{*} 

7ViU{*} 



, the Conley index with respect to H; 



h{'H,£; S) := jv ufmlglu-f*!- ' Conley index with respect to H and £; 



h{£;S) 



JViU(JVn£)u{*} 
Nn£u{*} 
JVin£u{*} 



, the Conley index with respect to £. 



The standard Conley index is able to analyse the dynamics at infinity in some situations, as 
illustrated in the following. 
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Example 2.5. A quadratic vector field under Poincare compactification 

We consider the following polynomial system in the plane: 



{ 



5{xiX2 " 1) 



(4) 



This system does not show any finite equilibria, hence also no finite periodic orbit. The dynamic 
at infinity was analysed in the vertical charts, and Figure ([2]) gives an overview of the global 
phase portrait on the Poincare hemisphere, seen from above. All equilibria at infinity are isolated 
invariant and their indices can be easily computed and read as follows: 



On the other hand, isolated invarianee seems to be too strong a requirement for the dynamics 
at infinity. Let us briefly motivate the next section through examples. Quadratic vector fields in 
the plane may exhibit accumulations of homoclinic loops at infinity as illustrated in the phase 
portrait 18, 39 of Figure Quadratic planar vector fields have been studied and classified: 
In [5] quadratic planar Hamiltonian vector fields are classified according to their full phase 
portraits on the Poincare hemisphere. There are 28 different phase portraits, 21 of them exhibit 
only isolated invariant equilibria at infinity, while 7 of them contain equilibria at infinity which 
are not isolated invariant because of an accumulation of homoclinic loops. Those 7 still fit our 
definition of complementary isolated invarianee introduced in Definition 13.11 
In [16) . quadratic planar vector fields are classified according to their phase portraits near the 
sphere at infinity. This classification contains 40 phase portraits: 25 of them show only isolated 
invariant equilibria at infinity, 12 of them show equilibria of isolated invariant complement, and 3 
of them show equilibria which are neither isolated invariant nor of isolated invariant complement. 
Figure 13] illustrates these three cases. The Phase Portraits 25, 18 are taken from Portrait 25 
exhibits only isolated invariant equilibria; Portrait 18 admits an equilibrium of isolated invariant 
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Figure 2: The Poincare compactification of the vector Field (Hj). 
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25 



18 



39 



Figure 3: Equilibria at infinity. 

complement at the bottom. The Phase Portrait 39 comes from jl6) and admits an equilibrium at 
the bottom which is neither isolated invariant nor of isolated invariant dynamical complement. 

For a polynomial vector field in finite dimensions, it seems that an accumulation of homoclinic 
loops on an equilibrium at infinity is structurally stable, as soon as this equilibrium is hyperbolic 
with respect to the flow within the sphere at infinity (as for instance in Figure Portrait 18). 
For more details, see Theorem 15.31 in Paragraph 15.21 which was proven in 0. To be able to deal 
with some of these situations, we develop in the following paragraph the concept of an invariant 
set of isolated invariant dynamical complement and the Conley index at infinity. 

3 Conley index at infinity 
3.1 Isolation of the complement 

The existence of an accumulation of homoclinic loops at infinity even for planar quadratic fields 
motivates the following definition of an invariant set of isolated invariant dynamical complement. 
The basic idea is the following. If it is not possible to isolate an invariant set S, try to isolate 
the invariant set Scomp containing every trajectory that does not tend to the first set S, neither 
in backward nor in forward time direction. In other words, Scomp contains the trajectories that 
remain far away from S and can therefore be considered as its dynamical complement. The 
precise definition is given in Definition p.ip . This idea is comparable to the concept of an 
attractor repeller pair, but here we do not require the connecting trajectories to run on a one- 
way-street. The compact neighborhoods isolating such an isolated invariant set Scomp are to be 
chosen arbitrarily large, as long as they do not intersect S itself. 

The complement of a set K is denoted by K'^ = T-L\K, not to be confused with the dynamical 
complement of a set S denoted by Scamp and defined in Definition p.ip . 

Definition 3.1. dynamical complement. 

Consider a normalized flow on the Poincarc Hemisphere %. Let S* be a closed (forward and 
backward) invariant set. We call the set Scomp 

Scomp := {x en-. a{x) nS = uj{x) n 5 = 0} 
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the dynamical complement of S. Furthermore the invariant set S is said to be of isolated invariant 
dynamical complement if Scomp is isolated invariant. 

The set Scomp is invariant because for every y £ o{x), a{x) ~ a{y) and uj{x) — uj{y). 

Example 3.2. Let us have a closer look at the phase portrait 18 of Figure ([3]). The equilibrium 
S at infinity at the bottom is not isolated because of the accumulation of homoclinic loops. The 
equilibrium S is of isolated invariant complement. Its dynamical complement Scomp consists in 

1. The repeller at infinity at the top, 

2. the saddle at the origin, 

3. the trajectory connecting the first to the second. 

The dynamical complement Scomp is invariant and isolated by every compact neighbourhood of 
itself that does not contain the equilibrium S. 



Determining the dynamical complement Scomp from S requires much knowledge on the global 
dynamics, so that one may want to formulate equivalent definitions of complementary isolated 
invariance where Scomp itself does not come into play. The first equivalent definition requires 
isolating properties for every large compact neighborhood that does not intersect S. The second 
one only for a continuous family of them, which is more easily verifiable in concrete examples. 

Proposition 3.3. We consider a normalized flow on the Poincare hemisphere. A closed invari- 
ant set S is of isolated invariant complement if and only if there exists a compact neighbourhood 
K d Ti with the following properties. 

1. The set K does not intersect the invariant set S: 

KnS^9>. 

2. The set K is an isolating neighbourhood in the Poincare hemisphere: 

Inv{K) C int-uiK). 

3. Every compact neighbourhood K' D K which does not intersect the invariant set S is also 
an isolating neighbourhood: 

K' compact neighbourhood ^ 

K' D K > ^ K' isolating neighbourhood. 

K'ns = 9 J 

Proof. Suppose that the dynamical complement Scomp of S is isolated invariant. Hence Scomp 
is closed as a maximal invariant set, disjoint from S, and there exists an isolating neighborhood 
K of Scomp that does not intersect S. We claim that this K also satisfies the third condition of 
Proposition 13. 31 Indeed, let K' be a compact neighborhood containing K with K' D S ~ 9, and 
X e Inv{K'). Then the orbit through x as well as its a— and limit sets, is contained in K' and 
therefore do not intersect S. By definition, such an x is in Scomp- Hence Inv{K') C Scomp- As 
K' D K , the reverse inclusion Inv{K') D Inv{K) = Scomp holds , and the claim Inv{K') = Scomp 
is proved. 

Now let us assume the existence of a compact neighborhood K satisfying the three conditions 
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of Proposition 13.31 First of all we prove that K as well as every superset K' as in condition (3) 
isolate the same invariant set. We proceed indirectly. If there exists a K' satisfying condition 
(3) with Inv(K') ^ Inv{K), then there is a point x G Inv{K') \ Inv{K), whose orbit o{x) is 
fully contained in K' but leaves K. Consider K = K U k, where fc is a compact neighborhood 
with {o{x) \ K) ndk ^ $ and k <Z K' - for example half of a small enough tubular neighborhood 
around o{x) \ K. Then o{x) C Inv{K) holds, but o{x) n iiT 7^ 0, so that K C K cannot be an 
isolating neighborhood which is a contradiction to condition (3). 

Now let us prove that Inv{K) = Scomp- As KCiS = 0, the inclusion Inv{K) C Scomp surely holds 
true. Now consider any point x S Scomp- If the point x were not in Inv{K), then it would also 
not belong to any Inv{K') for K' of condition (3), because we just proved Inv{K) = Inv{K'). 
Taking a sequence Kn of such sets so that d{Kn, S) — >■ 0, we get a sequence of points x„ S o{x) 
with the property 

^ Kn, d{Xn, S) 0. 

As H is compact, the sequence (a;„) converges (up to subsequence) to a point ^ G S*. The 
point ^ belongs to the orbit of x, or to its a- or w-limit set, contradicting x G Scomp- Hence 

Inv{K) ^ Scomp- □ 

Example 3.4. Let us illustrate this proposition with the phase portraits of Figure ([3]). 

Portrait 39: The equilibrium at the top is not of isolated invariant complement. If K'^ is a 
small enough neighborhood of the upper equilibrium, then K is not an isolating neighborhood: 
there are trajectories arbitrarily near this equilibrium that produce internal tangencies to duK 
but never leave K, hence Inv{K) <f_ int-uij^)- 

Portrait 18: The bottom equilibrium is of isolated invariant complement. If K is any hig 
enough compact neighborhood that docs not contain the bottom equilibrium, then every point 
of d-}iK leaves K in forward or backward time direction. Hence Inv{K) C int-u{K) i.e. the 
compact set K is an isolating neighborhood. Choosing K to small leads to violation of condition 
(3) as seen in Remark 13.81 

The third equivalence is the most easily verifiable in concrete examples. It "only" requires a 
continuous family of isolating neighbourhoods whose complements shrink on the complementary 
isolated invariant set S. But in fact it turns out to be a stronger requirement than that in 
Proposition p.3p . as we will explain in Remark (j3.8p . so that we have to exclude some pathologic 
cases to keep the equivalence. 

Definition 3.5. wild. 

We call a set S wild if it does not admit any compact neighborhood V such that 5 is a deformation 
retract of V. 

Remark 3.6. This definition of wild may not be completely standard, but this is the condition 
we need to guarantee the direction =^ in the equivalence below. An hawaiian earring is wild in 
the sense of Definition 13.51 

Proposition 3.7. We consider a flow (j) on the Poincare hemisphere. Let be S an invariant 
set that is not wild. Then S is of isolated invariant complement iff there exists a compact 
neighborhood V of S and a deformation retraction 

from V to S such that for every A G [0, 1[, the compact set Kx :~ clu {H{X, V)'') is an isolating 
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neighborhood in the Poincare hemisphere %. 

V.T e V, 
Vx £ V, 
Vx e S, 



The map H being a deformation retraction means: 

H{0,x) = X, 
H{l,x) e S, 
H{1, x) = X, 



Remark 3.8. The fact that 5 is a deformation retract of its neighborhood V iniphes in particular 
that they are both of the same homotopy type. It is not possible for wild sets, therefore we have 
to exclude them. To be more precise, let us consider an invariant hawaiian earring in phase 
portrait 18: S shall now be the union of the bottom equilibrium with an infinite but countable 
number of homoclinic loops of size tending to 0. This set S is an invariant set of isolated 
invariant complement Scomp (the same Scomp as in Example I3.2p for which wc cannot find any 
continuous family of isolating neighborhoods as defined in Proposition 13. 71 because the invariant 
set S considered is wild. 

If we do not assume that S is not wild in Proposition 13. 7[ then we only have 

Existence of a continuous family K\ as in 13.71 Scomp isolated invariant 

On the other hand, the requirement is strong enough to prevent the modification of the invariant 
set isolated by the K\ all along the deformation. Of course we will prove this below, but let 
us consider again S =bottom equilibrium of phase portrait (18), and refer again to Example 
13.21 Suppose we consider a continuous family of isolating neighborhoods with Kq =union of two 
disjoint squares around the repeller at the top and the saddle at the origin (hence yet to small to 
isolate the whole Scomp)- As the K\s grow, the two squares collide and form a rectangle around 
the orbit connecting the top and origin equilibria. The invariant set isolated by the continuous 
family changes without violating the isolation property. This does not contradict Proposition 
13.71 because this continuous family is not admissible there: S is not a deformation retract of 
V := cI-h{K^), because V is not simply connected. 

Proof. Assume the existence of a continuous family of isolating neighborhoods. First we prove 
that for all A S [0, 1), the sets Kx = d'n{H{X, Vy) isolate the same maximal invariant set. For 
that proceed indirectly: if there were an orbit o{x) through x € Kq and a A > with 

o{x) C K~^, 
o{x) ^ A'o ^ oix) nv ^9 

The set {A G [0, A]/ o{x) n -ff (A, V) ^ 0} contains and admit A as an upper bound, so that it 
admits a suprcmum 

Ai := sup{A e [0,A]/ o{x) n H{X,V) <J)}, < Ai < A (5) 

Claim 1: \ '^^^^ , where = cI-h{H{Xi,V)''), hence cannot be an isolating 

[o{x) ndKx, ^ 

neighborhood. 

Ifo(a;) ^ , then there exists a point y G int'n{H{Xi,V))r\o{x). But then y G int-u{H {X2,V))n 
o{x) for A2 slightly bigger than Ai, contradiciting the maximality of Ai ([5]). Hence o(x) C K\^. 
Now let us show o{x) f) dKx^ ^ 0. By definition ([5]) of Ai, there exists a sequence {^k)kefi ^ 
and sequences of points {xk)ken^ {Vklkm with 

Xk = i?(Aifc, Vk) e o{x) n H{nk, V), Vk G V. 
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The sequence {yk)ki£N converges by compacity of V, up to a subsequence, to a point y £ V . By 
continuity of H, we have 



H{fik,yk) = Xk (E o{x) — > H{Xi,y) E H{\i,V), as fc -> oo. 

This provides a point H{\i,y) £ o{x) Ci {K\^ n H{Xi, V)) = o{x) fl dK\^, as claimed. 

Now we prove that the common isolated invariant set T to all K\, A € [0, 1), is the dynamical 

complement Scomp of 5*. First, T C Scomp because T dV^ implies a{x) f] S ~ l>j{x) f] S = $ for 

every x £ T. Furthermore proceed again indirectly: if Scomp ^ T = hiv{K\) for all A G [0, 1), 

then 

VAe[0,l), 3 

o{xx)r\H{\V)^%. 

Considering a sequence A — > 1, we get as a limit a nontrivial intersection S H Scomp, which is a 
contradiction. Hence Scomp — T = Inv{Kx) for all A G [0, 1), is isolated invariant. 
Now we prove the second part of the equivalence. Assume that Scomp is isolated invariant. By 
Proposition 13.31 there exists a compact neighborhood K isolating Scomp with the property (3) 
of Proposition 13.31 Because S is not wild, there is a deformation retraction _ff of a compact 
neighborhood V C K'' to S which provides a continuous family of isolating neighborhoods 
Kx := cln{H{\, Vf, A e [0, 1), isolating Scomp- □ 

Definition 3.9. Conley index at infinity. 

Consider a flow tj) on the Poincare hemisphere, and an invariant set 5 C f at infinity of isolated 
invariant complement. Its dynamical complement Scomp is well defined and isolated invariant. 
We define the Conley index at infinity of S. denoted by h{S), as the classical Conley index of 
Scomp computed under the flow 4'-^{t, .) :— t, .) with reversed time direction. 

Remark 3.10. Let us justify why this provides a well defined index. 

A compact set R C H is isolated invariant under </> if and only if R is isolated invariant 
under 0_. Furthermore, If B is an isolating block for R with respect to the flow 0, and B± its 
immediate entrance and exit sets respectively, then B is also an isolating block for R with respect 
to the flow 0_ with respective entrance and exit sets Bzp. By classical Conley index theory, we 
know 



(B, _B_|_) is an index pair for R with respect to the flow (/)_, 

the Conley index (R) of R under the flow (j)^ is defined by the quotients 



h(j}^{n;R) := 
h^_{R) : { hcj)^{n,£;R) 
h<P_{£;R) := 



B 

B 

B+U(Bn£) 

Bns 



B+n£ 



and this definition does not depend on the choice of the index pair. 
By Definition 13.91 of the Conley index at infinity, we have 

hif,{S) '.= {Scomp)- 

The Conley index at infinity of S does not depend on the choice of the index pair for Scomp- 
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Example 3.11. The indices in Example p.2p . Figure ([3]), portrait 18, are the foUowing: 

• The rcpeller r at infinity at the top has index 

f h{n;r) = 

• The saddle O at the origin has index h(0) ~ S^. 

• The isolated invariant set Scomp has Conley index 

{h{T~L] Scomp) = 
h{J~L^ S'j Scomp) — 
h[£' Scomp) ^ 

• The Conley index at infinity of the degenerate equilibrium S reads 

rh{n;S)=0 
h{S) = <h{n,£;S)=J:' . 
[h{£;S) = SO 

Remark 3.12. The author is convinced that the concept of invariant set of isolated invariant 
complement, together with the concept of dynamical complement, are new and crucial for the 
study of dynamics at infinity or more generally on bounded manifolds with boundary. However, 
it is not clear wether the Definition p.9[) is the most sensible for the index of a set with isolated 
invariant complement. An argument against it is the following. If S is both isolated invariant and 
of isolated invariant complement, then both h{S) and h{S) are defined, but we see in Example 
(|3.13p that they do not have to coincide - basically because the summability formula fails for h. 
We want to point out here that this technical point is not crucial for the detection of heteroclinics 
to infinity - which is the goal that we pursue. 

Definition (j3.9p is useful for results on existence/nonexistence of connections to infinity. Such 
theorems are given in the next paragraph dealing with general properties of invariant sets of 
isolated invariant complement and their indices. 

Example 3.13. We consider a flow on the 3-dimensional Poincare hemisphere, which we repre- 
sent as a 3-dimensional ball whose boundary is the sphere at infinity. This fiow is sketched in 
Figure (|4]). The finite dynamics exhibit only an equilibrium at the origin which is a saddle. The 
1-dimensional stable manifold of the origin connects to two fully unstable equilibria at infinity. 
The 2-dimensional unstable manifold of the origin contains trajectories swirling toward a periodic 
orbit at infinity which is fully stable. Furthermore, on the sphere at infinity the two unstable 
equilibria are also connected to the stable periodic orbit by swirling trajectories. 
Let us call the periodic orbit in the sphere at infinity S. The invariant set S is isolated invariant. 
Its dynamical complement Scomp is well defined, being the union of the three equilibria and the 
stable manifold of the origin. Moreover, Scomp happens to be isolated invariant too. Therefore 
we can compare h{S) and h{S) in this example. A block B for Scomp is given by a cylinder 
around the unstable manifold of the origin connecting the three equilibria. The immediate exit 
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Figure 4: S-dimensional Poincare hemisphere. 

set consists of the whole boundary = dB. Therefore 

r h{n; Scornp) = r h{n; S) - SO 

h{Scomp) = < f; Scamp) = h{S) = /l_ (Scamp) - < £; 5) = 

i 5eo.mp) = S2 V £2 [ ; 5) = SO V s". 

Now let us consider the classical Conley index h{S) of the periodic orbit at infinity. A block D 
for S is given by a 3-dimensional ring around S, intersected with the Poincare hemisphere. The 
periodic orbit S being stable, the immediate exit set of the block D is empty and 

(h{H;S) = 0* 
h{S) = <^ hiU, £; Scamp) =0 
[hi£;S) = 0*, 

where 0* denotes the disjoint union of a 1-dimensional circle and the universal point not to 
be confused with S^ where the union is not disjoint. 

Hence h{S) ^ h{S). The topologies of the sets S and Scamp are different and this fact is reflected 
in the indices h{S) and h{S). 

3.2 General properties 

The Conley index at infinity is of course also defined on the homology and cohomology levels as 
the homology/cohomology of the pointed spaces given by h. The Poincare duality translates in 
terms of time duality and provides the following relationships between the indices. The proof of 
this proposition is a direct application of the result of |13j . 

Proposition 3.14. Consider a flow on the n-dimensional Poincare hemisphere %. We denote by 
H<, and H* the homological and cohomological Conley indices, and by and H* the homological 
and cohomological Conley indices at infinity, respectively. If S is of isolated invariant complement 
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Furthermore, the index at infinity allows us to formulate existence or nonexistence results. 
For this, we describe which types of Conley indices characterize attractors or repellers (Definition 
I3.15P in order to formulate the existence/nonexistence Theorem 13. 161 

Definition 3.15. Consider a flow on the Poincare hemisphere TL. We refer to an index coming 
from an index pair of the form {B, 0) as an "index of an attractor" an index . Such an index 
takes the form: 

h{n;S) = [B]U{*} 
h{'H,£;S) = 

h{£;S) = [Bn£]U{*} 

We refer to an index coming from an index pair of the form {B, SB) as an "index of a repeller" 
an index . Such an index takes the form: 

h{n,£;S) 
h{£;S) 

If an invariant set S is complementary isolated and its dynamical complement Scomp has the 
index h(Scomp) of an attractor or of a repeller, then S itself has a Conley index at infinity h(S) 
of a repeller or of an attractor respectively, due to the time reversal in the definition. 

This concept allows us to formulate the following existence/nonexistence theorem. 

Theorem 3.16. Consider a complementary isolated invariant set S <Z £ in the sphere at infinity. 
Its dynamical complement Scomp is isolated invariant and h{S) is well defined. 

1. If h{S) is the index of a repeller, then 

• there are trajectories S — ?• Scomp, or more precisely 

3x eTL\{S U Scamp), a{x) n 5 7^ 0, and Uj{x) C Scomp 

• there is no trajectory Scamp S. 

2. If h{S) is the index of an attractor, then 

• there are trajectories Scamp "> 5*, or more precisely 

3x eTL\{S[J Scamp), a{x) C Scamp, and uj{x) n S* 7^ 



Hri — k(^^, Scomp) 
Hn — k(T~^, £ , Scomp) 
Hn~k{£ , Scomp) 



= 

' B ' 

Bn£ 

d-HBr\£ 
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• there is no trajectory S Scomp- 

3. If h{S) is neither the index of an attractor nor of a repeller, then there are both trajectories 
S — )- Scomp and Scomp S, or more precisely 

Bx eJiXiS U Scomp), 0;{x) r\S ^<I}, and Uj{x) C Scomp 

3y (E H\{S U Scomp), a{y) C Scomp, and uj{y) n 5 7^ 

Proof. Suppose h{S) is of a repeller: by definition, the dynamical complement Scomp admits an 
index pair of the form [B, 0), where B is an isolating block for Scomp- Then for all x G B, the 
forward trajectory through x remains in B because the exit set of B is empty. Hence the w-limit 
uj{x) lies in the maximal invariant set Inv{B) = Scomp- 

On the other hand, for all a; G _B \ Scomp, there exists an entry-time T < such that 

\ft < T, (j){t, x) i B. 

Such trajectories accumulate on 5*, otherwise, they would entirely lie in Scomp by definition of 
the dynamical complement. 

A similar argumentation proves the two remaining claims. □ 

Remark 3.17. Of course, a very similar theorem holds if the set S is isolated invariant: 

1. If h{S) is the index of a repeller, then there are trajectories through x ^ S with a{x) C S, 
but no trajectories through x ^ S with lu{x) fl 5 7^ 0. 

2. If h{S) is the index of an attractor, then there are trajectories through x ^ S with u!{x) C S, 
but no trajectories through x ^ S with a{x) n S* ^ 0. 

3. If h{S) is neither the index of an attractor nor the index of a repeller, then there are both 
trajectories through x ^ S with a{x) C S and through y ^ S with Lu{y) C S 

The following example illustrates how such theorems may be used. 

Example 3.18. We consider the phase portrait as illustrated in Figure ([S]), which appears in 
the classification theorem of [2] ■ This portrait shows a fixed point S at infinity at the bottom of 
the phase portrait, which is of isolated invariant complement Scomp- The dynamical complement 
Scomp in Figure ([5]) is of trivial Conley index, hence h{S) = too. This index is neither the 
index of a repeller nor the index of an attractor. Hence there are trajectories with S in their 
a- limit sets, and trajectories with S in their w-limit sets, according to Theorem 13. 161 
Note that in the context of Conley index at infinity, a trivial index is no obstacle to the detection 
of connecting orbits. The connection structure will be analyzed in Example (j4.9D . 



4 Detection of Heteroclinics to Infinity 
4.1 Fundamental property 

The concept of isolated dynamical complement shows an interesting property on which our 
method for the detection of connection to some nonisolated invariant sets at infinity is based. 
Roughly speaking, this fundamental property says that leaving an isolating neighbourhood of 
Scomp forces leaving all of them - leading to an accumulation on S. More precisely, the following 
holds. 
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Figure 5: Phase portrait of Example (|3.18p 



Proposition 4.1. Let S d £ be an invariant set of isolated complement Scomp, cind N any 
isolating neighbourhood of the dynamical complement Scomp- Consider a trajectory o{x) under 
the flow (j) on the Poincare hemisphere % which connects to Scomp in backward or forward time 
direction but leaves N at a finite time to . Then this trajectory accumulates on S in forward or 
backward time direction respectively. In other words, if x G \ Scomp admits a to such that 
(/){to,x) ^ N, then: 

{Cx{x) C Scomp J l-^j{x) C Scomp 

^uj{x)r\S^(D \^a{x)r\S^lli 

Proof. Consider a growing sequence of isolating neighbourhoods (iV„)„gN of Scomp with HnGN ^\ 
Nn = S. Since Inv{N) = Scomp = Inv{Nn) for every n G N, it holds for any x E H that 

(j){to,x) ^ iV ^ Vn e N, 3t„/(t){tr,,x) i 7V„. 

Since the sets H \ Nn shrink on S as n grows, c?(0(i„, x), S*) — > as n oo. 

If in addition a{x) C Scomp, then the sequence (i„)neN tends to +oo, and, up to a subsequence, 

(l){tn,x) converges to a point of S, so that uj{x) n S' 7^ 0. 

Similarly, uj(x) C Scomp implies a(x) n 5* 7^ 0. □ 

Remark 4.2. Note that wc can only guarantee a nonempty intersection of the uj- or a-limits 
with the invariant set S of isolated invariant complement. An inclusion would not hold in general, 
since the uj- or a-limit could be a heteroclinic cycle containing equilibria in S and equilibria in 
Scomp, for example, or even a homoclinic loop to S. 

4.2 Construction of an Ersatz Infinity 

We wish to prove the existence of connections between a given invariant set S' C £ at infinity 
with isolated invariant complement Scomp and an isolated invariant set Q C Scomp- To this end, 
we replace 5* with an isolated invariant ersatz for which Conlcy index techniques apply and use 
the fundamental property 14.11 stated above to conclude the existence of a connection to S. 
We present here a construction of an ersatz which is realizable in many concrete cases. However, 



16 



we do not pretend that it is optimal. It may be useful in some cases to choose a more appropriate 
ersatz infinity. The fundamental property allows us to be very flexible at this point: as long as 
we keep the flow unchanged in an isolating block for Scomp, we may complete the flow as we 
please outside of this block. 

Let us give first an overview of the construction before we go into the technical details: 

1. Find an isolating block B for Scomp- Its boundary splits into immediate entrance and exit 
sets dB = £?+ U B^ , where 5+ n B^ contains only a finite number of points of exterior 
tangency to B. 

2. Choose retractions of B'^ and B~ to sets 6+ and b~ that are, typically, less complicated 
(for example a union of single points). 

3. Glue pieces of trajectories outside of B that follow the retractions. The sets are isolated 
invariant, and repellers/attractors respectively. 

Construction of the extended phase space. 

Let us fix an invariant set S in the sphere at infinity with isolated invariant complement Scomp- 
Furthermore, let B be an isolating block for Scomp- There exist strong deformation retracts 
r± : [0, 1] X B"^ — > B"^ of the entrance and exit sets to subsets C B^ , i.e. 

1. For all X g r±(0,x) = x. 

2. For all x £ b"^ and s G [0, 1], r='=(s, x) = x. 

3. For all x € B^ , r±(l,x) e 6± 

Furthermore we require that the retractions be injective: 

Vs G [0,1[, Va-i,a;2 G B^ , r^{s,xi) = r'^{s,X2) xi = X2- (Inj) 

This injcctivity property will be needed to construct a flow from the retracts without trajectories 
crossing each other. However this is no real restriction, as we see in the next lemma. 

Lemma 4.3. Let B be an isolating block for the isolated invariant dynamical complement Scomp 
of S <Z £ . Then there exist strong deformation retracts satisfying the injcctivity condition (Inj). 

Proof. Take 5± = B± and r±(s, .) = Id. □ 

After we have fixed one pair of deformation retracts with property (Inj), we extend the block 
B with subsets of [0, 1] x B^ defined by the following: 

/3± :={(s,r±(s,x)),sG [0,1],.TGB±}. (6) 

We now "glue" the pieces on the boundary of B by means of the equivalence relation 

X — y OT 

Vx, y G B U /3+ U /3", X - y if <( x = (0, e /^"^ U /S" and y ^ ^ e dB or 

y = (0, e 13+U and x^^edB. 



The extended phase space is defined as 



B\jB+\jp- 

n := —. (7) 
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Figure 6: Extended phase portrait for portrait 18 of Figure (|3]). 

For an illustration of the extension procedure on phase portrait 18 of Figure ([3]), see Figure (|6]). 

Construction of the extended flow 

We define a flow (j> on the extended phase space ik which foUows the flow (/> on the block B, and 
follows the deformation retracts after leaving the block B. We first have to fix some notation. 
For each point x € B, there exist entrance and exit times T-{x) G [— cxd,0], T+(x) € [0, +oo], 
defined as follows: 

T+{x) sup{t ^ 0/(f>{[0,tlx) C B}, (8) 
r_(.T) := inf{i =^ 0/<p{[t,0],x) C B}. (9) 

Note that T±[x) may be infinite. In case they are finite, 

(j){T_{x),x) e B+, and 

(j){T+{x),x) e B-. 

The maps T± : _B — >■ M U cxj are continuous because B is an isolating block. 
On the other hand, if we consider a point (s,y) G where s G [0,1[, then there is a point 
X G B^ such that y = r^{s,x), and this point is unique by virtue of the injectivity condition 
(Inj). To {s,y) G (3^, s ^ 1, we associate the unique Y^{y) := (r='=(s, .)) (y). 



Definition 4.4. For an initial condition x G B, the extended flow <j>{t, x) is defined as follows: 

U{t,x)&Bifte[T^{x),T+{x)], 
$it,x)^ I (l-e'^+(^)-*,r-(l-e^+(^)-*,(/)(T+(x),a;))) e p- for t ^ (10) 
[(1 -e*-^"("),r+(l -e*-^-("),(/.(T_(x),x))) G /3+ for t r_(x) 

In particular T^{x) = — oo implies (j){t,x) = (j){t,x) G B for all t ^ 0, and T^{x) = +oo implies 
(l){t,x) = (j){t,x) G B for all t ^ 0. Therefore, Invj^{B) = Inv^{B) = Scomp- 
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For an initial condition {s,y) G /?~, s G [0,1[, the associated point Y^{y) e with 
y = r~{s,Y~{y)) belongs to B so that 4){t,Y~{y)) is well defined through dTO]) for aU ^ e M. 
Hence we can define {s,y)) by the following: 

^^''^^'^^^^{^(/-ioga-\),y-(,))if.e^ 

For an initial condition {s,y) e s G [0,1[, the associated point Y^{y) e _B+ with 
y r+(s,y+(?;)) belongs to B so that 0(t, r+(y)) is well defined through ^ for ah t G M. 
Hence we can define (f>{t, {s,x)) by the following: 

= = ' ^ (12) 

"^^'^"'^^ \0(iog(i-s)-t,r+(y)) if,se [o,i[ ^ ^ 

Again Figure ^ shows how the flow is extended for the phase Portrait 18 of Figure 
The construction of the map guarantees the following: 

Proposition 4.5. The map : R x ?^ — ?► H defined by \4-4\ is a continuous flow. In other words, 
it fulfills the following properties: 

1. The map (j) is continuous. 

2. (f>{0, .) = id^ 

3. For allt,T e M and p £ H, 4>{t, 4>{t,p)) 4i{t + T,p). 
Remark 4.6. By abuse of notation, we denote by the sets 

&± = { (l,r±(l,x)) ,x e B^} C n, 

and call them ersatz infinities, because they replace the invariant set S C £ and are isolated 
invariant so that the Conley index is able to deal with them. By Definition (|4.4p of the extended 
fiow, they consist only of equilibria. Hence 

VxeB+,a^{x)^ lim x) = (l, r+(l, x)) £ 6+, 

\fx e B~, LU:{x) ^ lim ${t,x) = (l,r~(l,a;)) £ 6". 
In particular, 6+ is a repeller, while b~ is an attractor. 
Remark 4.7. Smoothing the extension 

After this construction, we end up with both a manifold and a flow which are continuous but 
not a priori smooth. The smoothness is desirable for technical parts of the Conley index theory: 
it provides the existence of blocks B which are manifolds with boundaries, and in particular 
guarantees the equality between the homology of the quotient and the relative homology 

H^,{B, B~). Therefore we should require the smoothing of both and 0. We conjecture that 
requiring the existence of a deformation smoothing both the extended phase space and the 
extended flow is no real restriction and is gcnerically fulfilled. This is at least the case in the 
examples we show here, although we do not stress this technical point for clarity. 
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4.3 Exhibit heteroclinics to infinity 



We have now prepared all the ingredients for the detection of heteroclinic orbits to an invariant 
set S at infinity of isolated invariant complement Scomp ■ There are many ways to formulate the 
detection of heteroclinics, for example in terms of connection matrices which are a powerful tool 
used to analyze the heteroclinic structure of a whole Morse decomposition. For simplicity, let 
us rather stick to the cornerstone of this theory here. For this, let us consider attractor repeller 
pairs in the extended phase space. More precisely, we assume that the extended phase space H 
contains an invariant set / (under the extended flow (/)) decomposable into an attractor repeller 
pair A,RcI with the following characteristics: 

1. One of the sets A, R is an invariant subset of U 6+. 

2. The other one is an invariant subset of Scomp- 

In this situation, the "nonidentity" h{I) h{A) V h{R) proves the existence of a heteroclinic 
connection i? — > A in the extended phase portrait (7^,0). As a consequence, there is an orbit 
leaving/entering the isolating block B of Scomp - under both the flows and (j) because they 
coincide in B. By virtue of Proposition 14.11 this orbit accumulates on S in the phase portrait 
(Hjcj)). This proves the following theorem. 

Theorem 4.8. Let (j) be a flow on the Poincare hemisphere %. Consider an invariant set S C £ 
of isolated invariant complement Scomp- Fix an isolating block B for Scomp o-nd proceed to the 
construction of the extended phase space Ti defined by ^ and the extended flow (j) on H defined 



Assume that there exist isolated invariant sets R and I under the extended flow cj) with 



such that (b , R) builds an attractor repeller decomposition of I , and suppose further that the 
Conley index detects a heteroclinic connection R ^ b^ by means of 



Then there is a heteroclinic connection R ^ S in the original flow (j) on %, or more precisely, 



such that (A, 6+) builds an attractor repeller decomposition of I , and suppose further that the 
Conley index detects a heteroclinic connection R ^ b^ by means of 



Then there is a heteroclinic connection S A in the original flow (j) on %, or more precisely, 



by gj. 



i? C S", 



I D Rub-, 



h{i)^h{R)y h{b-). 




h{i) ^ h{A)Wh{b+). 



there exists x ^ B with 




20 



b+ 



Figure 7: Extension and blocks for Example (|4.9p . 



Example 4.9. Let us proceed to the extension for the Example p.lSp whose phase portrait is 
illustrated in Figure ([5]). The bottom equilibrium 5* is invariant of isolated invariant dynamical 
complement Scomp- We choose an isolating block for Scomp by cutting straight through the 
accumulation of homoclinic loops. The Figure ([7]) shows the extended phase space for this 
example together with isolating blocks i?i^2 for two attractor repeller pairs involving the ersatz 
infinity 6+ and b~ . 

For the pair (6~ , r) , block Bi , the indices read: 



h{n,d'H;Inv{Bi)) = 
h{n;Inv{Bi)) =0 
h{diL]Inv{Bi)) = S" 



K(7i;r) = I]i 
{h{d'H]r) =0 



r h{n,d'H;b-') =0 
< h{n;b-) = SO 
[h{dn;b-) = SO 



As h{H; Inv{Bi)) = ^ V = h{'H] r) V h{'H; b-), we have by Theorem gJl) a heteroclinic 
orbit r ^ S 



For the pair (a, &+), block B2, the indices read: 

'h(H,dn;Inv{B2)) =0 ( h{H, dH; a) = Y.^ 
< Inv{B2)) = Si < a) = 

h{d'H\Inv{B2)) = T} [ a) 



h{n,dn;b+) = S2 
/iCH;6+) =0 
h{dn;b+) = Si 



As a?^; Inv{B2)) = ^ S^ V = 6+) V (9-H; a), we have by Theorem (gl 

a heteroclinic orbit 5 — >■ a 



Example 4.10. Let us exhibit heteroclinic connections to the bottom equilibrium in phase 
Portrait 18 of Figure The Figure ^ shows the extended phase space for this example 
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Figure 8: Extension and blocks for Portrait 18, Figure 



together with isolating blocks Bi -2 for two attractor repeller pairs involving the ersatz infinity 
and b~ . 

For the pair (c, block Bi, the indices read: 

h{H,dn;Inv{Bi)) ^0 ( h{H, dH; c) ^ Y.^ ( hiH,dn;b+) ^ ^"^ 
h{H;Inv{Bi)) ^T.^ <h{H;c)^T.^ <h{n;b+)=0 

h{dn\Inv{Bi)) = Y.^ \h{diL-c)=Q [ 6+) = 

As hin, dU] Inv{Bi)) = 7^ V = hiii, dH; b+) V h{H, dH] c), we have by Theorem 
a heteroclinic orbit S ^ c 

For the pair (6~,c), block ,62 , the indices read: 

'h{H,dn;Inv{B2)) ( h{n,dn;b-) =0 

< h{n; Inv{B2)) = < h{n; b~) = E" V 2° 

^ h{dn; Inv{B2)) 2° V S° \h{d'H;b-) ^ YP V YP 

As h{H]Inv{B2)) = E" 7^ S" V S" V El = /i(?i;c) V h{H]b-), we have by Theorem (gS]) a 
heteroclinic orbit c ^ S 



5 Conclusions 

5.1 Properties and Limits 

The construction of the extended phase portrait {H, (j)) depends on the choice of the isolating 
block B of the dynamical complement Scomp- In particular, the choice of B influences the 
topology of the ersatz infinity 6^, hence the ability of the Conley index to detect heteroclinic 
connections. Nevertheless it is sufficient to find one block B for which Theorem (j4.8p applies and 
detects a connection to the set S of isolated invariant dynamical complement Scomp- 
The dependence of the topology of b^ on the choice of B forbids us from defining the Conley 
index at infinity of S as the sum h^{b'^) V h^{b^): this sounds like a good option but docs not 

provide a well defined index h{S). 
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The connections to the ersatz infinity occur in a one way street; this was not the case for 
the invariant 5* itself, and this is the reason why it is not directly accessible to classical Conley 
index techniques. 

We described in the previous section how to detect heteroclinic orbits in the context of an 
attractor-repeller decomposition. More generally, Conley index and connection matrix theory 
allow one to analyze connection structure of a Morse decomposition involving more than two 
sets. Therefore it is natural to ask if we are able to deal with finitely many pairwisc disjoint 
invariant sets (S'i)i<ji^„ C f of isolated invariant complements (»5'i^^,^p)i^i<j„. The rough idea 
is to build a Morse decomposition in the extended phase space, analyze this decomposition via 
connection matrices and conclude the existence of heteroclinic orbits in the original flow on the 
Poincare hemisphere H. We do not want to go into the details of the proofs here and refer the 
reader to [5], Proposition (3.5.39) and below. 

Proposition 5.1. Consider a flow 4> on the Poincare hemisphere %, admitting a finite collection 
of pairwise disjoint invariant sets (S'i)i^i^„ C £ of isolated invariant complements iSi^^^^^)i<^i<^n- 
Let {Ki^)i<^i<^n be a collection of continuous families of isolating neighborhoods of the (>S'i^„„p)i^i^7 
as in Proposition ^3. 7| ). Then 

1. The disjoint union [J"^i C £ is an invariant set of isolated invariant dynamical complement 

n 
i=l 

2. There exists Xq G [0, 1[ such that K\ := cl-u (HiLi ^'^^wC-^ja )); ^ ^ -^o? forms a continuous 
family of isolating neighborhoods for n"=i ^iaomp ^'^ sense of Proposition {3.7^ . 

To proceed to the construction of the extended phase portrait {'H,(p)^ we need an isolating 
block B for n"=i ^iaamp- This block B should fulfill some extra conditions allowing us to analyze 
the connection structure to the individual Si's via their ersatz bf's. More precisely we need that 
the immediate entrance/exit sets B^ of B split into Bf leading after extension to ersatz bf of 
Si. For this purpose, we require isolating blocks Bi of Si^^^^ satisfying the following condition: 

V* 7^ G {1, ■ • ■ , n}, dn{Bf) n d„(Bj) = (13) 

Condition ([T51) means that the blocks Bi and Bj are big enough so that their complements Bf 
and neighborhoods of Si and Sj respectively, are well separated. The existence of "arbitrarily 
small" isolating blocks of Si^^^^ , Sj^^^^ is surely true, but not of "arbitrarily large" ones in general. 
Therefore the following 

Conjecture 5.2. As soon as the Si's are pairwise disjoint invariant sets of isolated invariant 
complements, it is possible to find a collection of isolating blocks {Bi)i^^i ... n} satisfying the 
condition 

Once we have a collection (_Bi)ig{i ... „| of isolating blocks for ('5'i^„„p)ig{i,... satisfying 
condition ([T3)). it follows that 

B :=f] B, 

1=1 

is an isolating block for nr=i ^ico^p- We build the extended phase portrait (H, (f) on this block 
B. Suppose the extended flow admits a Morse decomposition and classical connection matrix 
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theory exhibits connections between an isolated invariant set M C (Xi=i ^iaomp ^^'^ ersatz 
infinity bf^. As a consequence, there exists a trajectory leaving/entering the block B through 
its boundary dnB — UlLi '^'H.Bi, where the union is disjoint by virtue of condition ([T5|) . Hence 
this trajectory leaves/enters also the block Big through its boundary. By Proposition (|4.1I) . we 
know that the original trajectory under the flow on H accumulates on Si^. This shows that 
our method to detect heteroclinics is compatible with connection matrix theory. Should the 
Conjecture (|5.2p fail, the construction of the extended phase space should be performed for each 
Bi individually: The truth of the Conjecture (|5.2p does not affect the main Theorem (|4.8p . only 
the efficiency of its application. 

As noticed in Remark (|4.6p . the ersatz bf are repellers, the b~ are attractors. They are hence 
maxima/minima of chains of the Morse ordering. 

Let us again consider the case of a single invariant set S of isolated invariant complement in 
order to describe the type of information that leads to the detection of heteroclinic connections. 
Note that since is a repeller, its homological Conley index 77* (6+) has a nontrivial iJ„(6+), 
where n is the dimension of our original phase space, say n = dim('H). A nontrivial connection 
map S : Hn{b~^) —J- Hn-i{A) for an isolated invariant A C Scomp detects a heteroclinic connection 
fe+ — > A in ii, hence S* — > A in 'H by Theorem (|4.8p . Furthermore, the topology of itself 
may produce other nontrivial homologies Hj{b^), so that other connection maps 5 : Hj{b^) — > 
Hj^i{A') may detect other heteroclinics S — ?► A'. 

Analogously, the ersatz b~ is an attractor, its homological Conley index on the 0-th level is 
nontrivial so that a nontrivial connection map S : Hi{R) — > Ho{b^) detects a connection R ^ S. 
Further nontrivial homology groups Hj {b~ ) of the homological Conley index of b~ and nontrivial 
connection maps 5 : Hj+i{R') — ^ Hj{b~) may detect further heteroclinic connections R' — >■ S. 

5.2 Genericity Questions 

The following theorem has been proved in [S] . It deals with genericity of hyperbolic equilibria at 
infinity in the class of polynomial vector fields. 

Theorem 5.3. Let X be the set of polynomial vector fields on R of degree smaller than c? G N. 
Furthermore for every f ^ X , let us denote by fg the vector field induced by f on the sphere at 
infinity £ after compactification and normalization. The set 

Hyp = {/ £ I fs admits only hyperbolic equilibria on £} 

is open and dense in X. 

In other words the equilibria at infinity are structurally stable (in X) as soon as they are 
hyperbolic in the sphere at infinity. An accumulation of homoclinic loops, if it takes place in 
the finite part H \ f of the Poincare hemisphere, is structurally stable. Therefore the field of 
application of our methods is not restricted to pathological cases. 

5.3 Perspectives for PDEs 

The concepts we have developped in this paper are promising for the study of blow up phenomena 
in partial differential equations. 

Compactification has already been used to analyze the dynamics of the global unbounded 
attractor for the asymptotically linear equation ut = An + g{u), where u belongs to an Hilbert 
space, g is sublinear, and A nice enough. See [3l|6| for details. In this example, grow up takes 
place and all equilibria at infinity admits classical Conley index in the sense of |15| . 
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For PDE's whose leading terms are homogenous of degree d ^ 1 (for instance u"^ or u'^~^Au, 
...) we do not expect classical Conley index theory to be able to analyze the connection structure 
of the global unbounded attractor. 

In order to be able to apply our methods, some requirements are needed. First of all, we 
need to reduce the problem to a finite dimensional situation. Let us briefly explain why. The 
classical Conley index has been adapted to infinite dimensional situations and succesfuUy applied 
to partial differential equations. We do not want to develop this theory here: for the details, 
see [15| . Let us illustrate our dilemma on a dynamical system in a infinite dimensional phase 
space, compactified on an infinite dimensional Poincare hemisphere T-l, admitting an invariant set 
S at infinity of isolated invariant dynamical complement Scomp- Suppose further that Scomp is a 
hyperbolic equilibrium with finite dimensional unstable manifold of dimension u. The classical 
Conley index in the sense of [T5] is well defined: 

1 . If the direction pointing into the Poincare hemisphere is stable, the Conley index reads 

{h(T-L', Scomp) — 
h(l-L, £ ; Scomp) ~ 
h{£] Scomp) — 



2. If the direction pointing into the Poincare hemisphere is unstable, the Conley index reads 

{h['H; Scomp) — 0, 
h[T~L^ S] Scomp) — 
h{^£] Scomp) — 



The Conley index at infinity h{S) was defined by time duality: in this case it would not be defined 
in the sense of [15j because Scomp has infinite dimensional unstable manifold under reversed time 
direction. Or, to see it in a different way, our definition applied formally would produce an index 
at infinity equal to an infinite dimensional pointed sphere of codimension u: such a sphere is 
homotopy equivalent to a point, i.e. a trivial index, which does not make any sense. The same 
can be said about the index of the crstaz infinity which is a rcpellcr: the Conley index of 6"*" 
under the extended flow (/), h^{b^), is not well defined, or absurdly trivial. Hence theorem 14.81 
cannot be applied directly in infinite dimensional context. 

For these reasons, we have to reduce the problem to a finite dimensional one. A good way of 
doing so is to prove the existence of an invariant manifold - the incrtial manifold - containing 
the global unbounded attractor. See |3j for such an approach: in this work, grow up phenomena 
are studied, and the connection structure of the attractor is analyzed via a j/-map type of tool. 
An alternative approach is to use the Conley Index introduced by Izydorek and Rybakowski in [7| : 
the Conley Index at infinity h{S) of the example above would make sense in the finite dimen- 
sional Galerkin approximations, as well as the indices of the ersatz infinities. Hence exhibiting 
heteroclinic connections for each finite dimensional Calerkin approximations via the extended 
flow would allow to conclude to the existence of heteroclinic connections in the original PDE. 
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